15381: ARTIFICIAL INTELLIGENCE (FALL 2014) Homework 2: Planning, MDPs, and Reinforcement Learning (Solutions)

Graduate Artificial Intelligence 15 - 780 Homework # 3 : MDPs , Q - Learning , & POMDPs \ Out on

Towards a Unified Theory of State Abstraction for MDPs

State abstraction (or state aggregation) has been extensively studied in the fields of artificial intelligence and operations research. Instead of working in the ground state space, the decision maker usually finds solutions in the abstract state space much faster by treating groups of states as a unit by ignoring irrelevant state information. A number of abstractions have been proposed and stu...

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options—closed-loop policies for takin...

Feature Reinforcement Learning: Part I. Unstructured MDPs

General-purpose, intelligent, learning agents cycle through sequences of observations, actions, and rewards that are complex, uncertain, unknown, and non-Markovian. On the other hand, reinforcement learning is well-developed for small finite state Markov decision processes (MDPs). Up to now, extracting the right state representations out of bare observations, that is, reducing the general agent...

Efficient Abstraction Selection in Reinforcement Learning

This paper introduces a novel approach for abstraction selection in reinforcement learning problems modelled as factored Markov decision processes (MDPs), for which a state is described via a set of state components. In abstraction selection, an agent must choose an abstraction from a set of candidate abstractions, each build up from a different combination ofions, each build up from a differen...